完全有监督的语义细分从密集的口罩中学习，这需要封闭设置的大量注释成本。在本文中，我们使用自然语言作为监督，而无需任何像素级注释进行开放世界细分。我们将提出的框架称为FreeSeg，在该框架上可以从训练训练型模型的原始功能图中免费获得。与零射击或开放集分割相比，freeSeg不需要任何带注释的掩码，并且可以广泛预测超出类无需监督的分段之外的类别。具体而言，FreeSeg从图像文本相似性图（ITSM）中获得了可解释的对比度图像预处理（ICLIP）的自由掩码。我们的核心改进是浓密ICLIP的平滑最小池，具有部分标签和像素的分割策略。此外，没有复杂的设计，例如分组，聚类或检索，很简单。除了简单性外，Freeseg的表现超过了以前的最先进的边缘，例如在同一设置中，MIOU在MIOU上的13.4％。

对比性语言图像预训练（剪辑）通过随时可用的自然语言监督学习丰富的表示。它可以改善下游视觉任务的一般性能，包括但不限于零射击，长尾巴，细分，检索，标题和视频。但是，据我们所知，尚未研究剪辑的视觉解释性。为了提供其预测的视觉解释，我们提出了图像文本相似性图（ITSM）。基于它，我们出人意料地发现，剪辑比前景更喜欢背景区域，并且对人类理解提出了错误的可视化。在实验上，我们发现魔鬼在汇总部分，其中不适当的合并方法导致一种称为语义转移的现象。为了纠正和提高可视化结果，我们提出了蒙版的最大池，并使用自我监督图像编码器的注意力图。同时，解释性任务和识别任务需要不同的表示。为了解决这个问题，我们提出了双重预测，以满足这一要求。我们将上述方法整合为可解释的对比度图像预训练（ICLIP）。实验表明ICLIP极大地提高了可解释性。例如，在VOC 2012数据集中，非平凡的改进分别为$ 32.85 \％$和$ 49.10 \％$。

长尾学习旨在应对在现实情况下严重的阶级失衡下统治训练程序的关键挑战。但是，很少有人注意如何量化表示空间中头等的优势严重性。在此激励的情况下，我们将基于余弦的分类器推广到von mises-fisher（VMF）混合模型，该模型被称为VMF分类器，该模型可以通过计算分布重叠系数来定量地测量超晶体空间上的表示质量。据我们所知，这是从分布重叠系数的角度来衡量分类器和特征的表示质量的第一项工作。最重要的是，我们制定了类间差异和类功能的一致性损失项，以减轻分类器的重量之间的干扰，并与分类器的权重相结合。此外，一种新型的训练后校准算法设计为零成本通过类间重叠系数来提高性能。我们的方法的表现优于先前的工作，并具有很大的利润，并在长尾图像分类，语义细分和实例分段任务上实现了最先进的性能（例如，我们在Imagenet-50中实现了55.0 \％的总体准确性LT）。我们的代码可在https://github.com/vipailab/vmf \_op上找到。

数据隐私和类不平衡是许多机器学习任务中的常态，而不是例外。一方面已经启动了最近的尝试，解决了从普遍的私人数据中学习的问题，另一方面是从长尾数据中学习的。但是，这两个假设在实际应用中都可能存在，而同时减轻这两个问题的有效方法仍在开发中。在本文中，我们专注于在流行的隐私保存联合学习（FL）框架的背景下使用长尾（LT）数据分布进行学习。我们在FL框架中使用不同的本地或全局长尾数据分布来表征三个方案，并突出相应的挑战。在不同方案下的初步结果表明，未来的实质性工作是更好地解决特定的联合长尾学习任务的高度必要性。

In this paper, we propose a robust 3D detector, named Cross Modal Transformer (CMT), for end-to-end 3D multi-modal detection. Without explicit view transformation, CMT takes the image and point clouds tokens as inputs and directly outputs accurate 3D bounding boxes. The spatial alignment of multi-modal tokens is performed implicitly, by encoding the 3D points into multi-modal features. The core design of CMT is quite simple while its performance is impressive. CMT obtains 73.0% NDS on nuScenes benchmark. Moreover, CMT has a strong robustness even if the LiDAR is missing. Code will be released at https://github.com/junjie18/CMT.

Knowledge graphs (KG) have served as the key component of various natural language processing applications. Commonsense knowledge graphs (CKG) are a special type of KG, where entities and relations are composed of free-form text. However, previous works in KG completion and CKG completion suffer from long-tail relations and newly-added relations which do not have many know triples for training. In light of this, few-shot KG completion (FKGC), which requires the strengths of graph representation learning and few-shot learning, has been proposed to challenge the problem of limited annotated data. In this paper, we comprehensively survey previous attempts on such tasks in the form of a series of methods and applications. Specifically, we first introduce FKGC challenges, commonly used KGs, and CKGs. Then we systematically categorize and summarize existing works in terms of the type of KGs and the methods. Finally, we present applications of FKGC models on prediction tasks in different areas and share our thoughts on future research directions of FKGC.

Few Shot Instance Segmentation (FSIS) requires models to detect and segment novel classes with limited several support examples. In this work, we explore a simple yet unified solution for FSIS as well as its incremental variants, and introduce a new framework named Reference Twice (RefT) to fully explore the relationship between support/query features based on a Transformer-like framework. Our key insights are two folds: Firstly, with the aid of support masks, we can generate dynamic class centers more appropriately to re-weight query features. Secondly, we find that support object queries have already encoded key factors after base training. In this way, the query features can be enhanced twice from two aspects, i.e., feature-level and instance-level. In particular, we firstly design a mask-based dynamic weighting module to enhance support features and then propose to link object queries for better calibration via cross-attention. After the above steps, the novel classes can be improved significantly over our strong baseline. Additionally, our new framework can be easily extended to incremental FSIS with minor modification. When benchmarking results on the COCO dataset for FSIS, gFSIS, and iFSIS settings, our method achieves a competitive performance compared to existing approaches across different shots, e.g., we boost nAP by noticeable +8.2/+9.4 over the current state-of-the-art FSIS method for 10/30-shot. We further demonstrate the superiority of our approach on Few Shot Object Detection. Code and model will be available.

Graph Neural Networks (GNNs) have shown satisfying performance on various graph learning tasks. To achieve better fitting capability, most GNNs are with a large number of parameters, which makes these GNNs computationally expensive. Therefore, it is difficult to deploy them onto edge devices with scarce computational resources, e.g., mobile phones and wearable smart devices. Knowledge Distillation (KD) is a common solution to compress GNNs, where a light-weighted model (i.e., the student model) is encouraged to mimic the behavior of a computationally expensive GNN (i.e., the teacher GNN model). Nevertheless, most existing GNN-based KD methods lack fairness consideration. As a consequence, the student model usually inherits and even exaggerates the bias from the teacher GNN. To handle such a problem, we take initial steps towards fair knowledge distillation for GNNs. Specifically, we first formulate a novel problem of fair knowledge distillation for GNN-based teacher-student frameworks. Then we propose a principled framework named RELIANT to mitigate the bias exhibited by the student model. Notably, the design of RELIANT is decoupled from any specific teacher and student model structures, and thus can be easily adapted to various GNN-based KD frameworks. We perform extensive experiments on multiple real-world datasets, which corroborates that RELIANT achieves less biased GNN knowledge distillation while maintaining high prediction utility.

This paper focuses on designing efficient models with low parameters and FLOPs for dense predictions. Even though CNN-based lightweight methods have achieved stunning results after years of research, trading-off model accuracy and constrained resources still need further improvements. This work rethinks the essential unity of efficient Inverted Residual Block in MobileNetv2 and effective Transformer in ViT, inductively abstracting a general concept of Meta-Mobile Block, and we argue that the specific instantiation is very important to model performance though sharing the same framework. Motivated by this phenomenon, we deduce a simple yet efficient modern \textbf{I}nverted \textbf{R}esidual \textbf{M}obile \textbf{B}lock (iRMB) for mobile applications, which absorbs CNN-like efficiency to model short-distance dependency and Transformer-like dynamic modeling capability to learn long-distance interactions. Furthermore, we design a ResNet-like 4-phase \textbf{E}fficient \textbf{MO}del (EMO) based only on a series of iRMBs for dense applications. Massive experiments on ImageNet-1K, COCO2017, and ADE20K benchmarks demonstrate the superiority of our EMO over state-of-the-art methods, \eg, our EMO-1M/2M/5M achieve 71.5, 75.1, and 78.4 Top-1 that surpass \textbf{SoTA} CNN-/Transformer-based models, while trading-off the model accuracy and efficiency well.

The development of social media user stance detection and bot detection methods rely heavily on large-scale and high-quality benchmarks. However, in addition to low annotation quality, existing benchmarks generally have incomplete user relationships, suppressing graph-based account detection research. To address these issues, we propose a Multi-Relational Graph-Based Twitter Account Detection Benchmark (MGTAB), the first standardized graph-based benchmark for account detection. To our knowledge, MGTAB was built based on the largest original data in the field, with over 1.55 million users and 130 million tweets. MGTAB contains 10,199 expert-annotated users and 7 types of relationships, ensuring high-quality annotation and diversified relations. In MGTAB, we extracted the 20 user property features with the greatest information gain and user tweet features as the user features. In addition, we performed a thorough evaluation of MGTAB and other public datasets. Our experiments found that graph-based approaches are generally more effective than feature-based approaches and perform better when introducing multiple relations. By analyzing experiment results, we identify effective approaches for account detection and provide potential future research directions in this field. Our benchmark and standardized evaluation procedures are freely available at: https://github.com/GraphDetec/MGTAB.